Projection system, projection adjustment program, and projection method

ABSTRACT

A projection system includes a projection apparatus configured to perform position measurement and projection on a target object. The projection apparatus includes: an invisible light projector configured to project measurement light of invisible light onto the target object; a light receiver configured to receive reflected light of the measurement light reflected from the target object; and a calculator configured to calculate position information of the target object based on the reflected light of the measurement light. The calculator is configured to perform a mask processing of limiting a part of a projection range in which the measurement light is projected.

TECHNICAL FIELD

The present disclosure relates to a projection system that projects avideo onto a target object, a projection adjustment program, and aprojection method used in the projection system.

BACKGROUND ART

A technology of projecting a video onto a target object such as a screenor a structure, that is, a so-called projection mapping technology isknown. A projection mapping system includes a system having an imagecapturing function. For example, Patent Literature 1 discloses a systemthat can simultaneously acquire a 3D shape of a subject and capture animage of the object with visible light.

Various projection systems that use a plurality of projectionapparatuses have been proposed for applications such as a large-screendisplay. As this type of projection system, there are a multi-projectionsystem in which a plurality of projection apparatuses are arranged inhorizontal and vertical directions and projection screens of theprojection apparatuses are displayed side by side to perform a largerscreen display, and a stack projection system in which projectionscreens of projection apparatuses are displayed in an overlapping mannerto improve brightness of the projection screens. For example, PatentLiterature 2 discloses a system that can easily operate individualprojector apparatuses or collectively operate all projector apparatusesby performing infrared communication among a plurality of projectorapparatuses.

CITATION LIST Patent Literature

Patent Literature 1: JP-A-2005-258622

Patent Literature 2: WO-A1-2011/001507

SUMMARY OF INVENTION Technical Problem

It is an object of the present disclosure to provide a projectionsystem, a projection adjustment program, and a projection method thatcan appropriately avoid an error region of a projection range whenmeasuring a target object in the projection system.

Solution To Problem

The present disclosure provides a projection system including aprojection apparatus configured to perform position measurement andprojection on a target object, wherein the projection apparatus includesan invisible light projection unit configured to project measurementlight of invisible light onto the target object, a light reception unitconfigured to receive reflected light of the measurement light reflectedfrom the target object, and a calculation unit configured to calculateposition information of the target object based on the reflected lightof the measurement light, and wherein the projection apparatus isconfigured to perform a mask processing of limiting a part of aprojection range in which the measurement light is projected.

Further, the present disclosure provides a projection adjustment programconfigured to perform a processing related to adjustment of a projectionoperation of a projection apparatus by a computer in a projection systemincluding the projection apparatus configured to perform positionmeasurement and projection on a target object, the projection adjustmentprogram being configured to: perform position measurement by causing theprojection apparatus of the projection system to project measurementlight of invisible light onto the target object, receiving reflectedlight of the measurement light reflected from the target object, andcalculating position information of the target object based on thereflected light of the measurement light; detect an error region where adefect occurs in a measurement result of the position measurement; andset, for a target projection apparatus, a mask region for a maskprocessing of limiting a part of a projection range at a time ofprojecting the measurement light by using a measurement result of theerror region.

Further, the present disclosure provides a projection method including:a step of causing a projection apparatus to perform a mask processing oflimiting a part of a projection range in which measurement light ofinvisible light is projected; a step of projecting the measurement lightonto the projection range partially limited by the mask processing; astep of receiving reflected light of the measurement light; a step ofcalculating position information of a target object positioned withinthe projection range based on the reflected light of the measurementlight; a step of determining a projection position of a content based onthe calculated position information of the target object; and a step ofprojecting the content onto the determined projection position.

Advantageous Effects of Invention

According to the present disclosure, the error region of the projectionrange can be appropriately avoided when measuring the target object inthe projection system.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a summary of a configuration andfunctions of a measurement projection apparatus according to the presentembodiment.

FIG. 2 is a diagram showing a schematic configuration of the measurementprojection apparatus according to the present embodiment.

FIG. 3 is a diagram showing an example of invisible light measurementpatterns according to the present embodiment.

FIG. 4 is a block diagram showing a functional configuration of themeasurement projection apparatus according to the present embodiment.

FIG. 5 is a time chart showing an example of operations of themeasurement projection apparatus according to the present embodiment.

FIG. 6 is a diagram showing an example of a configuration of aprojection system according to the present embodiment.

FIG. 7 is a block diagram showing a functional configuration of aprojection adjustment apparatus according to the present embodiment.

FIG. 8 is a diagram showing a first example of setting operations ofmask regions in projection ranges of the projection system according tothe present embodiment.

FIG. 9 is a diagram showing a second example of a setting operation of amask region in projection ranges of the projection system according tothe present embodiment.

FIG. 10 is a flowchart showing a first example of a projectionadjustment method by the projection adjustment apparatus according tothe present embodiment.

FIG. 11 is a flowchart showing a second example of the projectionadjustment method by the projection adjustment apparatus according tothe present embodiment.

FIG. 12 is a flowchart showing a third example of the projectionadjustment method by the projection adjustment apparatus according tothe present embodiment.

FIG. 13 is a diagram showing a first example of a display screen by theprojection adjustment apparatus according to the present embodiment.

FIG. 14 is a diagram showing a second example of the display screen bythe projection adjustment apparatus according to the present embodiment.

FIG. 15 is a diagram illustrating a setting example of a mask region ofa plurality of measurement projection apparatuses of the projectionsystem according to the present embodiment.

DESCRIPTION OF EMBODIMENTS

[Introduction to Contents of Embodiment]

When it is considered that a video content is projected onto a targetobject to be projected, such as projection mapping, it is required toalign the video content on the target object as intended and project thevideo content. Finally, it is necessary to acquire geometric positioninformation of the target object viewed from a coordinate system of aprojection apparatus.

When projection is performed on a static target object, pre-measurementmay be performed only once separately from the projection. In that case,interference between projection and measurement can be ignored. On theother hand, it is considered to perform projection with no error in realtime on a target object that dynamically moves and/or deforms based on aresult of 3D measurement while performing the 3D measurement on thetarget object. In that case, it is required to perform measurement so asnot to influence a video content being projected.

However, the above-described Patent Literature 1 merely discloses that,by projecting a pattern image for the 3D measurement with invisiblelight, measurement can be performed without being influenced by visiblelight from a visible light source installed at another location.According to the technology of Patent Literature 1, only measurementresults conforming to a coordinate system of an image capturing devicecan be acquired.

In a field of measurement, in addition to the above-described PatentLiterature 1, for example, systems disclosed in Reference Non-PatentLiterature 1 and Reference Patent Literature 3 are known.

[Reference Patent Literature 3] JP-A-2013-192189

[Reference Non-Patent Literature 1] “Development of a 3,000-fps 3DImaging System Using a High-Speed Projector”, Proceedings of the 2007JSME Conference on Robotics and Mechatronics, “1P1-M02 (1)”-“1P1-M02(4)”, 2007 May 11

Reference Non-Patent Literature 1 discloses a method for measuring a 3Dshape at a high speed by using a light pattern projection. Themeasurement system of Reference Non-Patent Literature 1 includes animage capturing apparatus and a projection apparatus including a lightsource, a lens, a mirror element or a liquid crystal element. The imagecapturing apparatus has a function of performing high-speed imagecapturing. For example, the image capturing apparatus can performhigh-speed image capturing at 6000 fps. The projection apparatus canproject a binary pattern having 1024×768 pixels at 6000 fps or more.

Reference Patent Literature 3 discloses a measurement system thatadjusts a video content based on image capturing data. The measurementsystem of Reference Patent Literature 3 includes an image capturingapparatus, a projection apparatus, and a calculation apparatus. Thecalculation apparatus performs image recognition of a projection targetbased on an image capturing result acquired by the image capturingapparatus. The calculation apparatus generates a video of a region suchthat a video content is projected onto the region where the projectiontarget is recognized. The projection apparatus projects the videocontent onto the projection target.

The above-described Reference Non-Patent Literature 1 merely discloses atechnical level for performing the 3D measurement at a high speed. Sincean image having several tens of frames is required to transmitcoordinate information of the projection apparatus, it has beenconventionally difficult to perform the 3D measurement of a movingobject at a high speed. The technology of Reference Non-PatentLiterature 1 is considered to be meaningful in that it suggests apossibility that measurement can be performed at a high speed.

However, Reference Non-Patent Literature 1 only discloses a technologyof a 3D measurement unit, and does not refer to any coordinate system ofthe projection apparatus. Further, Reference Non-Patent Literature 1refers to an offline processing after high-speed image capturing, thatis, a processing in non-real time. Incidentally, in a computerarchitecture apparatus such as a personal computer that is premised onperforming image processing at 60 Hz or the like, a delay of severaltens of milliseconds or more occurs in input and output. As a result, itis difficult to capture an image of a moving object while projecting avideo onto the moving object and feed back the result to the projectionin real time.

According to the technology of the above-described Reference PatentLiterature 3, parallax is generated when positions of the imagecapturing apparatus and the projection apparatus are different from eachother. However, Reference Patent Literature 3 does not refer to anysolution to the parallax and does not refer to an increase in a speed ofthe system.

In view of such a situation, the inventor of the present application hasconceived a projection system that includes an invisible lightprojection device that can perform high-speed projection of invisiblelight such as infrared light, a visible light projection device that canperform high-speed projection of visible light, and an image capturingdevice that can perform high-speed image capturing, that can measure aposition of a target object with high accuracy by performing projectionand image capturing of measurement light by using pattern light ofinvisible light at a high speed, and that can perform projection byaligning a video content of visible light on the target object asintended.

Here, a projection system is assumed in which a plurality of projectionapparatuses that measure a position of a target object by projectingmeasurement light at a high speed are arranged. In such a projectionsystem, it is required to adjust a projection time and a projectionrange by the plurality of projection apparatuses, so that positionmeasurement of the target object and video projection can be performedwith high accuracy. The above-described Patent Literature 2 merelydiscloses that presence of another projection apparatus can be detectedand the plurality of projection apparatuses can be operated byperforming infrared communication among the plurality of projectionapparatuses.

In a projection system that performs position measurement and video projection of a target object by using a plurality of projection apparatusesas described above, when projection ranges of the plurality ofprojection apparatuses overlap each other, interference may occur inmeasurement light in an overlapping region, and accurate positionmeasurement may not be performed. Further, when a plurality ofprojection apparatuses that can perform highly accurate positionmeasurement are arranged, it may not be possible to easily grasp apositional relationship of a plurality of projection ranges such as anarrangement of projection ranges of the projection apparatuses andoverlapping of the projection ranges.

Accordingly, in the projection system that uses the plurality ofprojection apparatuses, when there is interference or an obstacle due toanother projection apparatus, there is a problem that an error occurs inthe position measurement and accurate position measurement cannot beperformed. Further, even when a single projection apparatus is used,there is a problem that accurate position measurement cannot beperformed due to an error when there are some obstacles. In view ofthese problems, it is desirable to appropriately avoid an error regionof a projection range by a method such as setting a mask region in theprojection range of the projection apparatus.

Hereinafter, each embodiment in which the configuration according to thepresent disclosure is specifically disclosed will be described in detailwith reference to the drawings as appropriate. However, unnecessarilydetailed description may be omitted. For example, detailed descriptionof a well-known matter or repeated description of substantially the sameconfiguration may be omitted. This is to avoid unnecessary redundancy inthe following description and to facilitate understanding of thoseskilled in the art. It should be noted that the accompanying drawingsand the following description are provided for a thorough understandingof the present disclosure by those skilled in the art, and are notintended to limit the subject matter recited in the claims.

Present Embodiment

As an example of the present embodiment, a projection system, aprojection adjustment program, and a projection method are exemplifiedin which, in the projection system that uses a plurality of projectionapparatuses, a mask region is set in a projection range of a targetprojection apparatus according to a predetermined condition, andaccurate position measurement can be performed by avoiding an errorregion.

(Summary of Measurement Projection Apparatus and Projection System)

FIG. 1 is a diagram illustrating a summary of a configuration andfunctions of the measurement projection apparatus according to thepresent embodiment. The present embodiment discloses an example in whicha position of a target object is measured using a measurement projectionapparatus 100 as shown in FIG. 1 as a projection apparatus that projectsa video onto the target object, and a video is projected according toposition information of the target object. Here, as a target object ontowhich a video is projected, a first target object 105 formed of a flator curved screen, a wall surface, or the like and a second target object106 formed of a person or the like positioned in front of the firsttarget object 105 are assumed. Hereinafter, the first target object 105and the second target object 106 may be simply referred to as the targetobjects 105 and 106. It is assumed that the second target object 106such as a person moves and moves each part of a body by performing adance or the like in front of the first target object 105 such as ascreen. That is, the target object 106 is in a state where a shape and aposition of each part are changed according to movement of the targetobject 106. Therefore, in order to project a predetermined video contentonto the target objects 105 and 106, it is necessary to measure aposition of the second target object 106 with respect to the firsttarget object 105 and acquire accurate position information of thetarget object 106.

The measurement projection apparatus 100 includes an image capturingdevice 101 as an example of a light reception unit, and a projectiondevice 122 that includes an infrared-light projection unit as an exampleof an invisible light projection unit that projects infrared light as anexample of measurement light of invisible light and a visible lightprojection unit that projects visible light, and that can projectinfrared light for measurement and visible light for video projection.The measurement projection apparatus 100 measures positions of thetarget objects 105 and 106 at high speed by projecting pattern light ofinfrared light whose projection coordinates are encoded by theinfrared-light projection unit of the projection device 122 at highspeed and capturing images of the target objects 105 and 106 at highspeed by the image capturing device 101. Details of the positionmeasurement of the target objects will be described later. Then, basedon position information of the target objects 105 and 106, themeasurement projection apparatus 100 projects a predetermined video bythe visible light projection unit of the projection device 122 in astate where the measurement projection apparatus 100 always performsalignment for a position of the moving target object 106 in particular.The measurement projection apparatus 100 may separately include aninvisible light projection device and a visible light projection deviceinstead of the projection device 122. In the present embodiment, it isassumed that a plurality of measurement projection apparatuses 100 areappropriately arranged to form a projection system as described later.

Examples of a projection system in which the plurality of measurementprojection apparatuses 100 are arranged include, for example, aprojection system that performs multi-plane projection on a targetobject to cover a large region, a projection system that performshigh-luminance projection by superimposed projection, a projectionsystem in which wrapping projection is performed by being disposed in acircumferential shape so as to surround a periphery of the targetobject, and the like. In such a projection system, interference occursin measurement light in a region where projection ranges overlap, andthere may be a problem that position measurement of a target objectcannot be accurately performed. In the present embodiment, an errorregion is avoided and the above problem is solved by setting a maskregion in a projection range of the target projection apparatusaccording to a predetermined condition such as a priority order.

(Configuration of Measurement Projection Apparatus)

Next, an example of a configuration and operations of the measurementprojection apparatus will be described in more detail.

FIG. 2 is a diagram showing a schematic configuration of the measurementprojection apparatus according to the present embodiment. Themeasurement projection apparatus 100 includes the image capturing device101, the projection device 122, and a calculation device 103.

In the present embodiment, the image capturing device 101 can performimage capturing at 6000 frames per second as in Reference Non-PatentLiterature 1. Further, the image capturing device 101 has a large-scaletransfer band without performing buffering therein, and can output imagecapturing data to the calculation device 103. Furthermore, the imagecapturing device 101 has sensitivity in an infrared light region.Hereinafter, on a premise of the above, an example of functions andoperations of devices will be described.

The projection device 122 is configured with an integrated projectiondevice including the infrared-light projection unit as an example of theinvisible light projection unit and the visible light projection unit.The infrared-light projection unit of the projection device 122projects, as an example of the measurement light, pattern lightindicating a pattern image obtained by encoding projection coordinatesdefined in a projection coordinate system. Further, the visible lightprojection unit of the projection device 122 projects video lightrepresenting video content. The projection device 122 may be configuredsuch that the invisible light projection device and the visible lightprojection device are separately provided. In the present description,the projection coordinate system means a coordinate system thatspecifies coordinates of each pixel of an image of video content that isa projected image projected from the visible light projection unit ofthe projection device 122. The coordinates that specify each pixel ofthe image of the video content are referred to as “projectioncoordinates” of the projection coordinate system. The projectioncoordinates also correspond to coordinates of each pixel of the patternimage projected from the infrared-light projection unit of theprojection device 122.

The projection device 122 includes a lens optical system 111, aninfrared LED light source 112, a display device 113, a visible light LEDlight source 114, and a dichroic mirror 115. The lens optical system 111may be configured with a single lens, or may be configured with aplurality of lenses (a lens group). The plurality of lenses can include,for example, a zoom lens and a focus lens.

The infrared LED light source 112 emits infrared light, which is anexample of invisible light, as the pattern light. The invisible lighthas, for example, a wavelength in an infrared light band (approximately700 nm to 1000 nm). In the present embodiment, the infrared LED lightsource is used as a light source of invisible light, but a light sourcethat emits ultraviolet rays can also be used.

The visible light LED light source 114 emits light in a visible lightband (approximately 380 nm to 780 nm) as video light. From a viewpointof simplification, the visible light LED light source 114 can be amonochromatic visible light source. However, it is needless to say thata full-color video may be projected by providing three light sources forthree colors of red, blue, and green. Further, if there is a color wheelthat can rotate at a sufficiently high speed, the full-color video canbe projected by providing a white light source such as a high-pressuremercury lamp instead of the visible light LED light source 114 andattaching the white light source to an output. Further, as a visiblelight source, it is possible to use a light source from which light canbe extracted for each wavelength from the high-pressure mercury lamp bya dichroic prism or the like. Accordingly, any light source can be usedin the present disclosure.

The display device 113 is, for example, an optical device such as adigital micromirror device (DMD) in which micromirrors are arranged on asquare of 1024×768, and generates a pattern image in which theprojection coordinates are encoded. The display device 113 can output avideo of 30000 frames per second in a binary pattern. The display device113 may be configured with a transmissive optical element instead of areflective optical element, or may also be replaced by a liquid crystaldevice.

The dichroic mirror 115 has characteristics of transmitting visiblelight and reflecting infrared light. As the dichroic mirror 115, a knowndichroic mirror can be widely used. A display device may be providedcorresponding to the infrared LED light source 112 and the visible lightLED light source 114, a dichroic prism may be provided instead of thedichroic mirror 115, and light from the two light sources and thedisplay device may be guided to the lens optical system 111.

The image capturing device 101 as an example of the light reception unitreceives the pattern light and captures an image to generate a capturedimage of the pattern light. The image capturing device 101 includes animage sensor, a lens optical system, and the like. For example, an imagesensor having a pixel count of 1024×768 can be used in correspondencewith the display device 113. In that case, if one pixel has a 8-bitresolution, a transfer band is about 38 Gbps. Here, it is assumed thatthe calculation device 103 is implemented by, for example, a fieldprogrammable gate array (FPGA). In consideration of a currentsemi-conductor technical level, the transfer band of about 38 Gbps iswithin a range that can be sufficiently implemented.

The image capturing device 101 has an image capturing coordinate system.In the present description, the image capturing coordinate system meansa coordinate system that specifies coordinates of each pixel of acaptured image acquired by the image capturing device 101. Indistinction from the “projection coordinates”, the coordinates of eachpixel of the captured image are referred to as “image capturingcoordinates” of the image capturing coordinate system.

The calculation device 103 as an example of a calculation unit decodes acaptured image into projection coordinate information indicatingprojection coordinates corresponding to image capturing coordinatesdefined in an image capturing coordinate system, converts the projectioncoordinate information into distance information to a target object withreference to the projection coordinate system, and selectivelydetermines a content of a video content according to the distanceinformation.

FIG. 3 is a diagram showing an example of invisible light measurementpatterns according to the present embodiment. FIG. 3 illustrates a partof encoded pattern images (coordinate patterns) corresponding to thepattern light. The pattern image shown in FIG. 3 is acquired bygray-coding an X coordinate and a Y coordinate of each mirror of thedisplay device 113 having 1024×768 micromirrors and then representingeach bit as a black-and-white binary image.

The infrared-light projection unit of the projection device 122 canproject the pattern light onto a target object 107 (corresponding to thetarget objects 105 and 106) based on, for example, a pattern image of1024×768 pixels. Both an X coordinate and a Y coordinate of a pixel arelarger than 512 and equal to or smaller than 1024. In that case, 10 bitsfrom bit0 to bit9 representing the X coordinate are gray-coded. Similarto the X coordinate, 10 bits from bit0 to bit9 representing the Ycoordinate are gray-coded. Accordingly, coordinate information can beencoded by allocating 10 bits to each coordinate, that is, 20 bits intotal. Hereinafter, an example of encoding 20-bit information by using40-frame image data will be described.

(X9 a) in FIG. 3 shows a pattern image corresponding to bit9 after the Xcoordinate is gray-coded. Further, in the present embodiment, since theprojection coordinates are encoded by Manchester encoding, an invertedpattern image obtained by bit-inverting bit9 is also used. (X9 b) inFIG. 3 shows an inverted pattern image obtained by inverting the imagepattern of (X9 a). Similarly, (X8 a) in FIG. 3 shows a pattern imagecorresponding to bit8 after the X coordinate is gray-coded, and (X8 b)shows an inverted pattern image obtained by inverting the image patternof (X8 a). (X7 a) in FIG. 3 shows a pattern image corresponding to bit7after the X coordinate is gray-coded, and (X7 b) shows an invertedpattern image obtained by inverting the image pattern of (X7 a).

(Y9 a) in FIG. 3 shows a pattern image corresponding to bit9 after the Ycoordinate is gray-coded. (Y9 b) in FIG. 3 shows an inverted patternimage obtained by inverting the image pattern of (Y9 a). Similarly, (Y8a) in FIG. 3 shows a pattern image corresponding to bit8 after the Ycoordinate is gray-coded, and (Y8 b) shows an inverted pattern imageobtained by inverting the image pattern of (Y8 a). (Y7 a) in FIG. 3shows a pattern image corresponding to bit7 after the Y coordinate isgray-coded, and (Y7 b) shows an inverted pattern image obtained byinverting the image pattern of (Y7 a).

Although not shown, there are pattern images and inverted pattern imagesrespectively corresponding to, for example, bits 6 to 0 of the Xcoordinate and the Y coordinate up to a measurable resolution. Theinfrared-light projection unit of the projection device 122 sequentiallyprojects 40 patterns including these patterns onto the target object107. The image capturing device 101 receives reflected light of patternlight from the target object 107, and sequentially captures projectedpattern images.

(Functional Configuration of Measurement Projection Apparatus)

FIG. 4 is a block diagram showing a functional configuration of themeasurement projection apparatus according to the present embodiment.The calculation device 103 has a function of controlling the entiremeasurement projection apparatus 100. The calculation device 103 can beimplemented by, for example, a computer, a calculation devicerepresented by a processor, or a semi-conductor integrated circuit. Thesemi-conductor integrated circuit is, for example, an applicationspecific integrated circuit (ASIC), an FPGA, or the like. Thecalculation device 103 may implement functions of components by using amemory in which a computer program that exerts the functions of thecomponents is installed and causing a processor in the semi-conductorintegrated circuit to sequentially execute the computer program.

The calculation device 103 includes an image input unit 401, a patterndecoding unit 402, a frame memory unit 403, a code decoding memory unit404, a coordinate conversion unit 405, a coordinate conversion memoryunit 406, a coordinate interpolation unit 407, a content generation unit408, a content memory unit 409, an image output unit 410, and a patterngeneration unit 411. Each memory unit in the calculation device 103 canbe configured with, for example, a RAM or the like.

FIG. 5 is a time chart showing an example of operations of themeasurement projection apparatus according to the present embodiment. Asshown in FIG. 5, the projection device 122 projects the pattern light inperiods 161, 163, and 165 and the video light in periods 162, 164, and166. That is, the projection device 122 projects the video light and thepattern light by time division multiplexing. “P” of the micromirror inthe drawing indicates a pattern image for measurement and “V” indicatesa video content as a projected image.

The pattern generation unit 411 turns on the infrared LED light source112 during the period 161. The pattern generation unit 411 generates apattern image for pattern projection by the method described above. Thepattern generation unit 411 outputs image data indicating a patternimage to the image output unit 410 so as to perform pattern projectionfor measurement on the display device 113. The image output unit 410outputs the image data from the pattern generation unit 411 andturning-on information of the infrared LED light source 112 to theprojection device 122 and the image input unit 401. Since the patternlight of the measurement light indicating the pattern image is projectedas the invisible light, the pattern light of the measurement light iscaptured and measured by the image capturing device 101, but the patternlight of the measurement light does not influence human vision.

The pattern generation unit 411 can output one pattern in 1/6000seconds. The pattern generation unit 411 outputs a total of 40 frames of10-bit coordinate images of the X coordinate and the Y coordinate andinverted images thereof during the period 161. On the other hand, theimage capturing device 101 captures an image at 40 frames insynchronization with a rate at which frames of the display device 113are output. In this example, a length of the period 161 is, for example,6.7 milliseconds.

The image output unit 410 outputs a pattern image to the projectiondevice 122 in synchronization with an output timing of the image data ofthe pattern generation unit 411. The projection device 122 projects thepattern image onto the target object. Further, the image input unit 401controls exposure of the image capturing device 101 in synchronizationwith an output timing of the pattern image of the image output unit 410.Accordingly, the image capturing device 101 captures a pattern image of40 frames.

The image input unit 401 receives a captured image (image capturingdata) of the pattern image captured by the image capturing device 101.The image input unit 401 transmits the received image capturing data tothe pattern decoding unit 402. The image input unit 401 determines apattern corresponding to the received image capturing data insynchronization with the image output unit 410.

The pattern decoding unit 402 decodes the captured image showing thepattern image from the image capturing device 101 into the projectioncoordinate information indicating the projection coordinatescorresponding to the image capturing coordinates defined in the imagecapturing coordinate system. Hereinafter, functions of the patterndecoding unit 402 will be described in more detail.

If the image capturing data received from the image input unit 401 is anon-bit inverted image of the X coordinate and the Y coordinate, thepattern decoding unit 402 writes the image capturing data in the framememory unit 403. If the image data is a bit-inverted image of the Xcoordinate and the Y coordinate, the pattern decoding unit 402 reads thenon-bit inverted image recorded in the frame memory unit 403 previouslyand obtains a difference between the two. Accordingly, the differencebetween the non-bit inverted image and the bit-inverted image isobtained, so that it is possible to distinguish between “0” and “1” ofprojection light without depending on a color of a projection target orambient light. A region where the difference is equal to or smaller thana predetermined value can be determined as a region where the projectionlight is not projected, and the region can be excluded from ameasurement target region.

The code decoding memory unit 404 is provided with a writing region foreach pixel of the image capturing device 101. The pattern decoding unit402 obtains the difference between the non-bit inverted image and thebit-inverted image, and then writes bit values of gray-coded coordinatedata in the writing region in bit units. A writing operation of thecoordinate data is performed for 40 frames during exposure time of theimage capturing device 101. Accordingly, information indicating whetherthe X coordinate and the Y coordinate of the projection device 102corresponding to pixels of the image capturing device 101 are presentand 10-bit values indicating the X coordinate and the Y coordinate whenthe X coordinate and the Y coordinate are present are written in thecode decoding memory unit 404. Finally, the pattern decoding unit 402reconverts the gray-coded coordinate data recorded in the code decodingmemory unit 404 into binary data and outputs the binary data to thecoordinate conversion unit 405.

With the above-described processing, it is possible to know from whichpixel of the projection device 122 the projection light captured at acertain pixel position of the image capturing device 101 is projected.That is, it is possible to know a correspondence relationship betweenthe projection coordinates defined in the projection coordinate systemof the projection device 122 and the image capturing coordinates definedin the image capturing coordinate system of the image capturing device101. Therefore, if a positional relationship between the image capturingdevice 101 and the projection device 122 is known, a distance to thetarget object can be acquired for each image capturing pixel bytrigonometry. However, the acquired information is distance informationcorresponding to image capturing pixels of the image capturing device101. Therefore, in the present embodiment, the distance information ofthe image capturing coordinates corresponding to the image capturingpixels of the image capturing device 101 is converted into distanceinformation corresponding to the projection coordinates of theprojection device 122.

The coordinate conversion unit 405 writes the data received from thepattern decoding unit 402 in a region of the coordinate conversionmemory unit 406 specified by an address corresponding to the projectioncoordinates of the projection device 122. Thereafter, the coordinateconversion unit 405 reads the distance information from the coordinateconversion memory unit 406 in an order of the X coordinate and the Ycoordinate of the projection device 122, thereby generating the distanceinformation corresponding to the projection coordinates of theprojection device 122.

At that time, a projection pixel having no corresponding point may begenerated. Specifically, in pattern images projected onto the targetobject, lights corresponding to a plurality of pixels can be captured byone image capturing pixel of the image capturing device 101. In thatcase, due to characteristics of gray-coding, the projection pixel havingno corresponding point is rounded to pixel coordinates of either of twoadjacent projection pixels, so that a projection pixel on one side hasno corresponding destination.

The coordinate interpolation unit 407 receives the distance informationcorresponding to the projection coordinates of the projection device 122from the coordinate conversion unit 405. The coordinate interpolationunit 407 interpolates distance information for projection coordinateshaving no distance information. This is performed by using aninterpolation method such as linear interpolation based on distanceinformation of peripheral coordinates only in a place where a certainnumber of projection coordinates having distance information that can beinterpolated exist in a periphery thereof. The coordinate interpolationunit 407 outputs distance information based on the projectioncoordinates to the content generation unit 408. As described above, byperforming reading of the captured image of the pattern image andcalculation of position information including distance information tothe target object, a high-speed position measurement operation can beperformed in real time.

The content generation unit 408 generates a video content for projectionacross the period 162 and the period 163. The content generation unit408 processes video content recorded in advance in the content memoryunit 409 based on the distance information received from the coordinateinterpolation unit 407, and outputs the processed video content to theimage output unit 410. Hereinafter, the processed video content may bereferred to as “processed video content” while being distinguished fromthe unprocessed video content recorded in advance.

The content generation unit 408 generates a video content that does nothave coordinate deviation and accurately corresponds to a distance tothe target object. Further, the content generation unit 408 canselectively determine a content of the video content according to thedistance information. For example, it is possible to perform processingssuch as cutting out and detecting only an object at a certain distanceand accurately drawing a video content for visible light projection. Thecontent generation unit 408 outputs the processed video content forprojection to the image output unit 410.

The image output unit 410 outputs a video content for visible lightprojection generated in the period 162 and the period 163 to theprojection device 122 in the period 164. The projection device 122 turnson the visible light LED light source 114, and projects video lightcorresponding to the video content by the display device 113. Thedisplay device 113 can output 30000 binary frames per second. Therefore,for example, it is possible to project an image of 256 gradations byusing 255 frames in 8.5 milliseconds. Since the projection is performedwith the visible light source, the projection is visually recognized bya human.

In the period 163, in parallel with the generation of the video contentfor the projection, projection and image capturing of the pattern imageby infrared light are performed in the same manner as in the period 161.The content generation unit 408 generates a video content that does nothave a coordinate deviation and accurately corresponds to a distance tothe target object across the period 164 and the period 165. Then, in theperiod 166, the projection device 122 projects the video content for theprojection. Accordingly, position measurement and projection can becontinuously performed.

A repeated cycle of measurement and projection is, for example, 15.2milliseconds if measurement time (the period 161) is 6.7 millisecondsand projection time (the period 162) is 8.5 milliseconds. This meansthat the cycle can be implemented with a throughput of 60 Hz or higher.Further, time from measurement to reflection of a measurement result(hereinafter, referred to as “delay time”) can be set to 15.2milliseconds, which is the same as that of the repeated cycle.Accordingly, since the throughput of 60 Hz or higher can be achieved, aflicker of a projected image due to non-display time such as ameasurement period in which the video content is not projected can besufficiently reduced to a level that is not noticeable to a human eye.In FIG. 5, the delay time corresponds to a total time of the period 162and the period 163.

In the measurement projection apparatus 100 of the present embodiment,by performing the video projection and the position measurement with thesame measurement projection apparatus, it is possible to preventoccurrence of a deviation between the projection and the measurement inprinciple, and to implement superimposition of geometric measurementthat does not interfere with a video of visible light. Further, if thecalculation device 103 can decode a pattern image captured by the imagecapturing device 101, the calculation device 103 can withstand arelative position measurement. Therefore, even when installationaccuracy is not sufficiently secured, the calculation device 103 canwithstand practical use. In this regard, simplicity of installation canbe secured. Further, high robustness can be acquired against an increasein an error of an installation relationship due to deterioration overtime.

In the calculation device 103, the pattern generation unit 411 performsa mask processing of limiting a part of a projection range in whichmeasurement light can be projected. At this time, the pattern generationunit 411 sets a part of a pattern image corresponding to the mask regionto be not transmitted and masks a partial region of the projectionrange. The projection device 122 blocks light of the mask region by thedisplay device 113, turns on the infrared LED light source 112 toproject the measurement light into the projection range partiallylimited by the mask processing.

(Configuration of Projection System)

FIG. 6 is a diagram showing an example of a configuration of theprojection system according to the present embodiment. The presentembodiment shows an example in which, in the projection system as shownin FIG. 6, a projection adjustment apparatus 200 configured with apersonal computer (PC) or the like is used to adjust projectionoperations of the plurality of measurement projection apparatuses 100,or to support adjustment work of a projection operation by a user. Theprojection system includes the plurality of (four in the illustratedexample) measurement projection apparatuses 100 and the projectionadjustment apparatus 200 that performs a processing related toadjustment of a projection operation of the measurement projectionapparatus 100. Here, it is assumed that a projection range is set suchthat parts of projection ranges of the measurement projectionapparatuses 100 overlap so as to cover a large region by performing themulti-plane projection on the target objects 105 and 106 by theplurality of measurement projection apparatuses 100. The illustratedexample shows a configuration that uses four measurement projectionapparatuses P1, P2, P3, and P4.

The projection adjustment apparatus 200 is connected to a monitor 250including a display for displaying information, and displays a displayscreen including various pieces of projection information for adjustinga projection operation on the monitor 250. The projection adjustmentapparatus 200 is configured with an information processing apparatussuch as a PC including a processor and a memory and executes apredetermined computer program, so that functions such as display ofprojection information and automatic adjustment of a projectionoperation are implemented.

FIG. 7 is a block diagram showing a functional configuration of theprojection adjustment apparatus according to the present embodiment. Theprojection adjustment apparatus 200 includes a processing unit 210, astorage unit 220, and a communication interface (I/F) 230. Theprojection adjustment apparatus 200 is connected to the measurementprojection apparatus 100 via the communication interface 230, andtransmits and receives various pieces of information such as settinginformation on a measurement operation, projection range information,mask region information, and target object position measurementinformation. The projection adjustment apparatus 200 is connected to adisplay unit 240 and an input unit 260, displays the display screen onthe display unit 240, and inputs an operation instruction from the inputunit 260. The display unit 240 is configured with a display device suchas the monitor 250 in FIG. 6. The input unit 260 is configured with aninput device such as a keyboard, a mouse, a touch pad, and a touchscreen(not shown).

The storage unit 220 includes a storage device including at least one ofa semiconductor memory such as a flash memory, a storage device such asa solid state drive (SSD) and a hard disk drive (HDD), and the like. Thestorage unit 220 stores a projection adjustment program 221 thatperforms a function related to adjustment of a projection operation.

The processing unit 210 includes a processor such as a centralprocessing unit (CPU) and a digital signal processor (DSP). Theprocessing unit 210 performs a processing according to the projectionadjustment program 221 and implements a function such as mask regionsetting 211.

The communication interface 230 is an interface for transmitting andreceiving information to and from an external apparatus such as themeasurement projection apparatus 100 by wired communication or wirelesscommunication. As a wired communication interface, for example, auniversal serial bus (USB), an Ethernet (a registered trademark), or thelike may be used. As a wireless communication interface, for example,Bluetooth (a registered trademark), a wireless LAN, or the like may beused.

As a function of the mask region setting 211, the projection adjustmentapparatus 200 performs setting related to a mask region in theprojection range of the measurement light based on a measurement resultby the measurement projection apparatus 100, a predetermined condition,and the like.

The measurement projection apparatus 100 acquires information of a setmask region and performs the mask processing of limiting the part of theprojection range in which the measurement light can be projected. Themeasurement projection apparatus 100 projects the measurement light tothe projection range partially limited by the mask processing by theprojection device 122, and receives reflected light of the measurementlight by the image capturing device 101. Further, the measurementprojection apparatus 100 calculates position information of the targetobjects 105 and 106 positioned within the projection range by thecalculation device 103 based on the reflected light of the measurementlight, and determines a projection position of a content based on thecalculated position information of the target objects. Then, themeasurement projection apparatus 100 projects the content to thedetermined projection position by the projection device 122.

A part or all of functions of the processing unit of the projectionadjustment apparatus 200 may be provided in the calculation device 103of the measurement projection apparatus 100, and may be performed suchthat a processing such as the mask region setting is performed in themeasurement projection apparatus 100.

(Setting Example of Mask Region)

Here, some examples of setting a mask region in the projection system ofthe present embodiment will be described.

FIG. 8 is a diagram showing a first example of setting operations ofmask regions in projection ranges of the projection system according tothe present embodiment. The first example shows a setting example in acase where there is interference with a projection range of anothermeasurement projection apparatus and there is an obstacle in theprojection range. As shown in an upper left part of the drawing, it isassumed that, regarding the measurement projection apparatus 100 ofinterest, there are an area er1 that overlaps and interferes with aprojection range PEx of another measurement projection apparatus and anarea er2 where erroneous measurement occurs due to an unintendedobstacle OB, in a projection range PE1 of the measurement light. Forexample, in a region where there is an unintended obstacle such as anobject having a mirror surface, a measurement error occurs due todiffused reflection. Further, also for a region that interferes withmeasurement light of another measurement projection apparatus, there isa high possibility that erroneous measurement occurs, and it isconsidered that the region is a region where an unintended measurementerror occurs in a broad sense.

In this case, the areas (error regions) er1 and er2 where a measurementerror occurs due to the diffused reflection caused by the obstacle orthe overlapping of the projection ranges are detected, mask regions me1and me2 corresponding to the error regions are set as shown in an upperright part of the drawing, and measurement lights of the mask regionsare blocked. That is, in each pattern image of the measurement light asshown in a lower left part of the drawing, the mask regions me1 and me2are set as shown in a lower right part of the drawing, and patternlights obtained by masking mask region portions are generated andprojected. Similar to a black-and-white pattern of the pattern image, amask processing of the mask region of the measurement light can beperformed by blocking the measurement light in pixel units by thedisplay device 113.

The mask regions me1 and me2 are shown to completely avoid theprojection range PEx of another measurement projection apparatus and theobstacle OB for easy understanding in the drawing, and it is preferableto set the mask regions to be small such that the projection range PE1is slightly applied to a boundary with another projection range PEx andthe obstacle OB or another projection range PEx and the obstacle OB.Accordingly, when performing the position measurement, a measurementrange can be appropriately overlapped at a boundary of the mask region,a connection portion with a position measurement result by anothermeasurement projection apparatus can be smoothly connected and measured,and thus the position measurement can be performed more accurately.

With the setting of the mask regions and the mask processing, it ispossible to appropriately mask a region where erroneous detection occursdue to the interference with the measurement light of anothermeasurement projection apparatus or the obstacle, and to perform normalposition measurement. The first example is applicable not only to a casewhere the position measurement and the video projection are performed bycontrolling the plurality of measurement projection apparatuses 100, butalso to a case where the mask processing is performed in the singlemeasurement projection apparatus 100 in an environment in which themeasurement projection apparatus 100 is used together with anothermeasurement projection apparatus that cannot be managed and controlledin an integrated manner, such as a different type of apparatus having adifferent specification. In this case, in the single measurementprojection apparatus 100, when an error occurs, an error region may beset as a mask region based on a measurement result.

FIG. 9 is a diagram showing a second example of a setting operation of amask region in projection ranges of the projection system according tothe present embodiment. The second example shows a setting example in acase where multiple projection such as the multi-plane projection isperformed using the plurality of measurement projection apparatuses. Asshown in an upper left part of the drawing, it is assumed that parts ofprojection ranges PE1, PE2, PE3, and PE4 of the measurement light by thefour measurement projection apparatuses (P1 to P4) 100 overlap eachother, and for example, there is the area er1 that interferes in theprojection range PE4. In this case, an area (an error region) er1 wherea measurement error occurs due to the interference is detected, theerror region is set as the mask region me1 as shown in an upper rightpart of the drawing, and the measurement light of the mask region isblocked. That is, in each pattern image of the measurement light asshown in a lower left part of the drawing, the mask region me1 is set asshown in a lower right part of the drawing, and pattern light obtainedby masking a mask region portion is generated and projected.

Here, when a plurality of projection ranges overlap as shown in theillustrated example, up to three projection ranges are allowed tooverlap, a mask region is set in one of the projection ranges for aregion having four or more overlaps, a part of the measurement light ismasked, and the number of overlapping projection ranges is set to 3 orless. Accordingly, the number of phases when the plurality ofmeasurement projection apparatuses are sequentially operated can bereduced, the number of divisions of position measurement and projectionby time division can be reduced to enable efficient position measurementand projection in a short time, the interference of the measurementlight can be prevented as much as possible to prevent a measurementerror, and highly accurate position measurement can be performed.

The mask region me1 is shown to completely avoid four overlaps with theprojection ranges PE1 to PE3 of other measurement projection apparatusesfor easy understanding in the drawing, and it is preferable to set themask region to be small, for example, to a boundary with otherprojection ranges PE1 to PE3, and to appropriately overlap theprojection ranges with three or less overlaps. Accordingly, whenperforming the position measurement, a connection portion with positionmeasurement results by other measurement projection apparatuses can besmoothly connected and measured at a boundary of the mask region, andthe position measurement can be performed more accurately.

The projection adjustment apparatus 200 determines an order, a phase, aprojection timing, and the like of projection operations of theplurality of measurement projection apparatuses in relation to themeasurement projection apparatuses 100 to be controlled based on anarrangement of the projection ranges and the like. Further, theprojection adjustment apparatus 200 can perform the position measurementby the measurement projection apparatuses 100 to be controlled,determine a mask region according to overlapping of the projectionranges and presence or absence of an obstacle, and notify themeasurement projection apparatuses to set the mask region. A specificexample of a processing of the projection adjustment program of theprojection adjustment apparatus 200 will be described later.

In the present embodiment, an error region of the position measurementcan be avoided by appropriately setting a mask region in the projectionranges of the measurement projection apparatuses 100 to perform the maskprocessing. Further, when the plurality of measurement projectionapparatuses are used, interference of the measurement light among themeasurement projection apparatuses can be prevented. Therefore,appropriate position measurement can be performed in the measurementprojection apparatus of the projection system. For example, whenperforming the position measurement and the video projection on a movingtarget object such as a dancer by using the plurality of measurementprojection apparatuses, it is possible to repeatedly perform the videoprojection while accurately measuring a position in real time. Further,when there are a fixed target object such as a screen and a movingtarget object such as a dancer, even in the projection system that usesthe plurality of measurement projection apparatuses, it is possible toaccurately measure positions of the target objects in real time, and toindividually generate and project a video content according to thepositions of the target objects.

(Operations of Projection Adjustment Apparatus)

FIG. 10 is a flowchart showing a first example of a projectionadjustment method by the projection adjustment apparatus according tothe present embodiment. The first example shows a schematic procedure ofa setting processing of a mask region by the projection adjustmentapparatus. Here, an example of a processing related to the mask regionsetting 211 by the projection adjustment program 221 is shown. The firstexample can be applied in any case of setting a mask region of a singlemeasurement projection apparatus and setting a mask region whencontrolling a plurality of measurement projection apparatuses to performthe position measurement and the video projection.

The projection adjustment apparatus 200 performs the processingaccording to the projection adjustment program 221 in the processingunit 210. First, the projection adjustment apparatus 200 transmits aninstruction to the measurement projection apparatus 100 to becontrolled, and causes the measurement projection apparatus 100 toproject the measurement light to perform the position measurement (S11).Next, the projection adjustment apparatus 200 selects the measurementprojection apparatus that sets the mask region (that is, performs themask processing) based on an operation input by the user (S12). At thistime, the projection adjustment apparatus 200 selects a measurementprojection apparatus that performs the mask processing according to apredetermined condition in consideration of various conditions such asdesignation of an apparatus based on user input, or a priority order ofapparatuses, an arrangement of a projection range, a resolution orluminance of the video projection, and an operation mode such as manualsetting or automatic setting. Then, the projection adjustment apparatus200 sets the selected measurement projection apparatus to 100,determines overlapping of projection ranges and presence or absence ofan obstacle by using a measurement result of step S11, and determines amask region for an error region (S13). The determined mask regioninformation is transmitted to the measurement projection apparatus thatperforms the mask processing to set the mask region.

According to the present embodiment, by setting the mask region in theerror region of the projection range of the target measurement projection apparatus according to a predetermined condition such as thepriority order, the error region of the projection range can beappropriately avoided when measuring a target object. Accordingly, it ispossible to accurately perform the position measurement of the targetobject.

FIG. 11 is a flowchart showing a second example of the projectionadjustment method by the projection adjustment apparatus according tothe present embodiment. The second example shows a procedure of asetting processing of a mask region when performing the positionmeasurement and the video projection by controlling the plurality ofmeasurement projection apparatuses. Here, an example of a processingrelated to the mask region setting 211 by the projection adjustmentprogram 221 shows a processing procedure of determining an overlappingstate of a plurality of projection ranges, determining a mask region,and displaying the projection ranges and projection timings of themeasurement projection apparatuses.

The projection adjustment apparatus 200 performs the processingaccording to the projection adjustment program 221 in the processingunit 210. First, as a user interface, the projection adjustmentapparatus 200 performs GUI operation display on the display unit 240,and displays an operation screen for a user operation (S21). Then, theprojection adjustment apparatus 200 performs the following processingaccording to a user operation. When the user gives an operationinstruction to perform the measurement, the projection adjustmentapparatus 200 initializes a counter value for counting the measurementprojection apparatuses as i=1 (S22), causes the i-th (1st in an initialstate) measurement projection apparatus to project the measurement lightto check whether an error occurs and records an error region when ameasurement error occurs (S23). Subsequently, the projection adjustmentapparatus 200 causes the i-th measurement projection apparatus toperform projector projection (S24). As the projector projection, theprojection of the measurement light described above is performed. Atthis time, the projection adjustment apparatus 200 causes cameras ofimage capturing devices in all the measurement projection apparatuses tocapture the projection of the measurement light by the i-th measurementprojection apparatus (S25).

The projection adjustment apparatus 200 determines, regarding aprojection range of the i-th measurement projection apparatus, aconnection relationship of the projection range such as presence orabsence of overlapping and which measurement projection apparatusoverlaps based on image capturing results of all the measurementprojection apparatuses, and records connection relationship information(S26). When there is overlapping of the projection ranges, an image ofthe measurement light is captured by other measurement projectionapparatuses. A connection relationship of projection ranges of theplurality of measurement projection apparatuses can be determined bypositions of the measurement projection apparatuses in which the imageof the measurement light is captured.

Next, the projection adjustment apparatus 200 sets the counter value ofthe measurement projection apparatus to i=i+1 (S27), and determineswhether the counter value i is smaller than the number of measurementprojection apparatuses (S28). When the counter value i is smaller thanthe number of measurement projection apparatuses (S28: Yes), that is,when there is the measurement projection apparatus for which thedetermination of the connection relationship of the projection rangeshas not been processed, the processings of steps S23 to S28 are repeatedin the similar manner as described above. That is, the projectionadjustment apparatus 200 checks the error region in order for all themeasurement projection apparatuses, causes the measurement light to beprojected, causes other measurement projection apparatuses to performthe operation of capturing the image of the measurement light, anddetermines the connection relationship of the projection ranges of themeasurement projection apparatuses. The measurement processing ofprojecting the measurement light by each measurement projectionapparatus and image capturing by all the measurement projectionapparatuses may be performed only when the user gives an operationinstruction to perform the measurement, or may be automaticallyperformed at predetermined time intervals.

When the counter value i is equal to the number of measurementprojection apparatuses (S28: No), the projection adjustment apparatus200 generates projection position information indicating the connectionrelationship of the projection ranges of the measurement projectionapparatuses. Here, as an example, a display screen of a graph displayshowing the connection relationship of the projection ranges is created(S29). Then, the projection adjustment apparatus 200 draws a graph on anoperation screen of the display unit 240, and displays the graph showingthe connection relationship of the projection ranges of the measurementprojection apparatuses (S30).

When the user gives an operation instruction for manual mask setting,the projection adjustment apparatus 200 selects a measurement projectionapparatus that performs the mask processing and an overlapping place ofa projection range of the measurement projection apparatus based on anoperation input by the user (S31). Then, the projection adjustmentapparatus 200 sets a mask region for the overlapping place of theprojection range of the selected measurement projection apparatus (S32).When there is an error region due to an obstacle or the like in ameasurement result of the measurement projection apparatus, a maskregion may be set in the error region without designation by the user.Information of the set mask region is transmitted to the targetmeasurement projection apparatus that performs the mask processing toset the mask region.

When the user gives an operation instruction for automatic mask setting,the projection adjustment apparatus 200 searches for a place whereoverlapping of the projection ranges is equal to or larger than aspecified value as a predetermined condition (S33). As the specifiedvalue of the overlapping, for example, 4 is used, and an overlappingplace of four or more projection ranges is searched. Subsequently, theprojection adjustment apparatus 200 selects a measurement projectionapparatus to be masked based on priority information indicating a presetpriority for the overlapping place equal to or larger than the specifiedvalue (S34). As the priority, a priority order of an apparatus set inthe system, an arrangement of projection ranges, a resolution andluminance of video projection, and the like are appropriately used. Forexample, a projection range is secured by giving priority to anapparatus whose projection range is close to a center, an apparatuswhose area of a projection range is large, an apparatus whose resolutionis high, an apparatus whose luminance is high, an apparatus whose lifeof a light source is long, an apparatus whose overlapping number islarge, and the like, and the mask processing is performed by othermeasurement projection apparatuses. Further, it is also possible to givepriority to an apparatus that is not under management and difficult tocontrol in the projection adjustment apparatus 200, such as a differenttype of apparatus having a different specification, an apparatus thatdoes not have a mask function, and the like, and to perform the maskprocessing in an adjustable measurement projection apparatus. Then, theprojection adjustment apparatus 200 sets a mask region such that thereis no overlapping place in the projection range of the selectedmeasurement projection apparatus (S3 5). When there is an error regiondue to an obstacle or the like in a measurement result of themeasurement projection apparatus, a mask region may be set in the errorregion regardless of an overlapping place of the projection ranges.Information of the set mask region is transmitted to the targetmeasurement projection apparatus that performs the mask processing toset the mask region.

FIG. 12 is a flowchart showing a third example of the projectionadjustment method by the projection adjustment apparatus according tothe present embodiment. The third example shows a procedure of amodified example of the second example. Here, an example of a processingrelated to the mask region setting 211 by the projection adjustmentprogram 221 shows a processing procedure of determining an overlappingstate of a plurality of projection ranges, determining a mask region,and displaying projection timings and a mask region of the measurementprojection apparatuses.

The projection adjustment apparatus 200 performs the processingaccording to the projection adjustment program 221 in the processingunit 210. First, the projection adjustment apparatus 200 transmits aninstruction to the measurement projection apparatus 100 to becontrolled, and causes the measurement projection apparatus 100 toproject the measurement light to perform the position measurement (S41).Next, the projection adjustment apparatus 200 checks the overlappingstate of the projection ranges and determines a connection relationshipof the projection ranges by, for example, the same processing as theprocessings of S22 to S28 in FIG. 11. Then, the projection adjustmentapparatus 200 generates projection position information indicating theconnection relationship of the projection ranges of the measurementprojection apparatuses. Here, as an example, a display screen of a graphdisplay showing the connection relationship of the projection ranges iscreated. Further, the projection adjustment apparatus 200 draws a graphon an operation screen of the display unit 240, and displays a graphshowing the connection relationship of the projection ranges of themeasurement projection apparatuses (S42).

Next, the projection adjustment apparatus 200 determines whether anoperation mode set by an operation input of the user or the like is anautomatic mode in which mask setting is automatically performed or amanual mode in which the mask setting is manually performed (S43). As auser interface, a GUI operation display may be performed on the displayunit 240 to display an operation screen for a user operation, and userinput in the automatic mode or the manual mode may be received.

In a case of the manual mode, the projection adjustment apparatus 200selects the measurement projection apparatus that performs the maskprocessing based on an operation input by the user (S44). At this time,information of a region of an overlapping place of a projection range ofthe measurement projection apparatus designated by the user input and ofan error region is acquired. Then, the projection adjustment apparatus200 determines a mask region of the projection range of the selectedmeasurement projection apparatus so as to avoid overlapping and an errorof the projection range (S45). When there is an error region due to anobstacle or the like in a measurement result of the measurementprojection apparatus, a mask region may be set in the error regionwithout designation by the user.

In a case of the automatic mode, the projection adjustment apparatus 200inputs desired number of phases of projection operations of theplurality of measurement projection apparatuses (S46) and inputspriority information of the measurement projection apparatuses (S47) asa predetermined condition. The number of phases and the priorityinformation may be input as, for example, set values as initial valuesset in advance, or may be acquired based on an operation input by theuser. Further, the number of phases and the priority order may bedetermined and input according to the connection relationship of theplurality of projection ranges acquired by the measurement processing instep S41. For number of phases, for example, 2 or 3 is used. As thepriority, a priority order of the apparatuses set in the system, anarrangement of the projection ranges, resolution and luminance of thevideo projection, a designation order according to the user, and thelike are appropriately used. Then, the projection adjustment apparatus200 selects the measurement projection apparatus that performs the maskprocessing based on the input number of phases and priority information(S48). Subsequently, the projection adjustment apparatus 200 determinesa mask region such that there is no overlapping place in a projectionrange of the selected measurement projection apparatus (S49). When thereis an error region due to an obstacle or the like in a measurementresult of the measurement projection apparatus, a mask region may be setin the error region regardless of an overlapping place of the projectionranges.

It is also possible to perform a combination of the manual mode and theautomatic mode. For example, a procedure may be in such a way that arecommended mask region is temporarily determined by an automatic modeprocessing, and the mask region is determined by selecting, partiallychanging, or adjusting a region based on user input by a manual modeprocessing, and the like.

Next, the projection adjustment apparatus 200 generates and displaysprojection information indicating the number of phases of themeasurement projection apparatus of the projection system, theprojection timings of the measurement projection apparatuses, the maskregion of the projection ranges, and the like (S50). As an example ofprojection information for display, the projection adjustment apparatus200 creates a display screen such as an image display in which theprojection ranges of the measurement projection apparatuses areindicated by a figure or the like, a graph display in which a connectionrelationship such as overlapping of the projection ranges is indicatedby a node, a connection line, or the like, a timing display in which theprojection timings of the measurement projection apparatuses areindicated, and an image display in which the mask region of theprojection ranges is indicated by a figure or the like. Then, theprojection adjustment apparatus 200 draws a character, a figure, agraph, or the like on the operation screen of the display unit 240, andperforms the above-described image display, graph display, timingdisplay, and the like.

Then, the projection adjustment apparatus 200 waits for an operationinput by the user to perform setting check, and checks whether aninstruction input of “OK” is received from the user (S51). When theinstruction of the setting OK is not acquired from the user, theprojection adjustment apparatus 200 returns to step S43 and performs theprocessings of steps S43 to S51 again. When receiving the instruction ofthe setting OK from the user, the projection adjustment apparatus 200notifies the measurement projection apparatuses of information of theset mask region and the set projection timings, causes the measurementprojection apparatuses to set the projection timings, and causes thetarget measurement projection apparatus that performs the maskprocessing to set the mask region (S52).

Accordingly, by detecting an error region by performing the positionmeasurement in one or a plurality of measurement projection apparatusesand setting a mask region in a projection range of the targetmeasurement projection apparatus by the manual setting or the automaticsetting, it is possible to appropriately avoid the error region of theprojection range when measuring the target object.

Here, some examples of a display screen on which the projection ranges,the projection timings, the mask region, and the like are displayed areshown.

FIG. 13 is a diagram showing a first example of the display screen bythe projection adjustment apparatus according to the present embodiment.The first example is an example of the display screen including an imagedisplay 301 showing the arrangement of the projection ranges of theplurality of measurement projection apparatuses and a graph display 302showing the connection relationship of the projection ranges. In theimage display 301, the arrangement of the projection ranges PE1 to PE4by the four measurement projection apparatuses P1 to P4 is schematicallyrepresented by a quadrangular graphic display. The overlapping place ofthe projection ranges may be highlighted in a distinguished manner by acolor, a pattern, or the like. Accordingly, the user can easily grasp aregion where the projection ranges overlap and an overlapping state bythe image display. In the graph display 302, positions of the projectionranges of the four measurement projection apparatuses P1 to P4 aredisplayed by nodes 321 such as circular marks, and are represented by agraph in which the nodes 321 where the projection ranges overlap and aconnection relationship exists are connected by connection lines 322.With the graph display, the user can check the connection relationshipof the projection ranges of the measurement projection apparatuses at aglance. Further, it is also possible to display and grasp overlappingnumbers, overlapping positions, and the like of the projection ranges.

FIG. 14 is a diagram showing a second example of the display screen bythe projection adjustment apparatus according to the present embodiment.The second example is an example of a display screen including a timingdisplay 311 showing the number of phases and projection timings of theplurality of measurement projection apparatuses and an image display 312showing a mask region of the projection ranges of the plurality ofmeasurement projection apparatuses. The timing display 311 shows thenumber of phases (the number of phases=2) of the projection systemincluding the four measurement projection apparatuses P1 to P4, andshows the projection timings of the measurement projection apparatusesP1 to P4 by a timing chart. The illustrated example shows an operationof performing time-division projection by using two phases by causingthe four measurement projection apparatuses to sequentially emit light.The user can easily grasp the projection timings of the measurementprojection apparatuses. In the image display 312, the mask region me1 isschematically represented by a graphic display in the projection rangesPE1 to PE4 by the four measurement projection apparatuses P1 to P4. Theprojection range including the mask region, that is, the projectionrange (PE4 in the illustrated example) in which the mask processing isperformed may be highlighted in a distinguished manner by a differentcolor, a different pattern, or the like. Further, priority informationof the measurement projection apparatuses and the projection ranges maybe displayed. Thereby, the user can easily check the mask region.

Further, the user can also set and adjust an order, the phases, theprojection timings, and the like of the projection operations of theplurality of measurement projection apparatuses based on display of theprojection position information.

With the display screen as in the above example, the user can easilygrasp a positional relationship of the projection ranges, the projectiontimings, the mask region, and the like of the plurality of measurementprojection apparatuses, and can provide display of the projectionposition information with good visibility. Therefore, it is possible toeffectively support adjustment work of the projection operations whenthe user sets the mask region of the measurement projection apparatuses.

(Setting Example of Mask region of Plurality of Apparatuses)

FIG. 15 is a diagram illustrating a setting example of a mask region ofthe plurality of measurement projection apparatuses of the projectionsystem according to the present embodiment. Here, an example ofdynamically setting the mask region when performing the mask processingin the plurality of measurement projection apparatuses is shown.

Here, in the projection system, it is assumed that there is anoverlapping place where the projection ranges PE1 to PE4 by the fourmeasurement projection apparatuses P1 to P4 overlap with each other andoverlap four times at the maximum. Then, a maximum value of overlappingnumbers of the overlapping place is set to 3, and the mask region is setsuch that the overlapping numbers are equal to or smaller than 3.Accordingly, a maximum value of the number of phases of the measurementprojection apparatus is 3, and the number of phases can be reduced. Inthis case, the mask region may be set in the projection range of any oneof the measurement projection apparatuses and masked.

The mask region is not limited to one that is fixedly set with respectto a projection range of a specific measurement projection apparatus,and it is possible to dynamically set the mask region by switching ameasurement projection apparatus that performs the mask processing. Forexample, as shown on a lower side of FIG. 15, a projection range(indicated by a broken line in the drawing) in which a mask region isprovided is switched in time series such as PE4→PE3→PE2→PE1, and themask region is allocated in a time division manner, so that ameasurement projection apparatus that performs the mask processing isrotated. In this case, in one measurement projection apparatus, maskingis performed at a rate of 1 out of 4 projections of the measurementlight. Accordingly, in the overlapping region of the projection ranges,it is possible to measure the overlapping place by all the measurementprojection apparatuses. For example, when there is a target object thatmoves across the overlapping region, it is possible to seamlessly andsmoothly track a position of the target object, and it is possible toimprove accuracy of position measurement.

(Another Setting Example of Mask Region)

As another setting example related to a mask region, when there is atarget object or an obstacle that moves in a projection range and anerror region is displaced, it is also possible to dynamically set themask region in response to a change in a position of the error regionand displace the region to be masked. In this case, by repeating aprocessing of measuring a position of the target object or the obstacleand setting the mask region based on a measurement result atpredetermined timings during a projection period, it is possible toperform the dynamically displaced mask region setting and thedynamically displaced mask processing.

Further, the mask region is not limited to a region where the maskprocessing is performed on the measurement light in a projection range,and it is also possible to perform the mask processing on videoprojection of the visible light. In this case, both the measurementlight and the visible light may be subjected to the mask processing inthe same mask region, or different mask regions may be set for themeasurement light and the visible light to perform the mask processing.

As described above, a projection system of the present embodiment is aprojection system including a measurement projection apparatus 100 as aprojection apparatus configured to perform position measurement andprojection on target objects 105 and 106. The measurement projectionapparatus 100 includes an invisible light projection unit configured toproject measurement light of invisible light onto the target objects, animage capturing device 101 as a light reception unit configured toreceive reflected light of measurement light reflected from the targetobjects, and a calculation device 103 as a calculation unit configuredto calculate position information of the target objects based onreflected light of the measurement light. The measurement projectionapparatus 100 includes, for example, a projection device 122 includingthe invisible light projection unit that includes an infrared LED lightsource 112 and a visible light projection unit that includes a visiblelight LED light source 114. The measurement projection apparatus 100 isconfigured to perform a mask processing of limiting a part of aprojection range in which the measurement light can be projected.Accordingly, it is possible to appropriately avoid an error region ofthe projection range when measuring the target objects in the projectionsystem.

The projection system includes a projection adjustment apparatus 200including a processing unit 210 configured to perform a settingprocessing for a mask processing of limiting a part of the projectionrange of the measurement light by the measurement projection apparatus100. The processing unit 210 projects the measurement light by themeasurement projection apparatus 100 to perform position measurement,detects an error region where a defect occurs in a measurement result ofthe position measurement, and sets, for a target measurement projectionapparatus 100, a mask region when projecting the measurement light byusing a measurement result of the error region.

Accordingly, by using the measurement result of the error region, forexample, it is possible to appropriately set a mask region such that themeasurement light is not projected onto the error region, and to performthe mask processing of limiting a part of the projection range.Therefore, the mask region can be appropriately set in the projectionrange and the measurement light of a portion of the mask region ismasked, so that the error area of the projection range can beappropriately avoided when measuring the target objects. Accordingly,the position measurement of the target objects can be accuratelyperformed.

In the projection system, the processing unit 210 detects a region wherea measurement error occurs due to an obstacle in the projection range asthe error region, and sets the mask region for the region. Accordingly,the mask processing can be performed using the region where themeasurement error occurs as the error region, and the positionmeasurement of the target objects can be accurately performed.

Further, in the projection system, the processing unit 210 detects aregion where measurement light from another measurement projectionapparatus overlaps in the projection range as the error region, and setsthe mask region for the region. Accordingly, the mask processing can beperformed using the overlapping region of the projection range as theerror region, and the position measurement of the target objects can beaccurately performed.

The projection system includes a plurality of measurement projectionapparatuses 100. The processing unit 210 is configured to cause a firstmeasurement projection apparatus of the projection system to project themeasurement light onto the target objects, cause a second measurementprojection apparatus of the projection system to receive reflected lightof the measurement light reflected from the target objects, determine aconnection relationship of a projection range of the first measurementprojection apparatus based on the received reflected light of themeasurement light, perform a determination processing of the connectionrelationship for all measurement projection apparatuses to be processed,detect, as the error region, a region where measurement light fromanother measurement projection apparatus overlaps in projection rangesof measurement projection apparatuses of the projection system, and setthe mask region for the region. Accordingly, in the projection systemincluding the plurality of measurement projection apparatuses, anoverlapping region of the projection ranges of the measurementprojection apparatuses can be detected, and a mask region can beappropriately set in a projection range of a target measurementprojection apparatus according to a predetermined condition such as apriority order. Therefore, in any one of the plurality of measurementprojection apparatuses, the measurement light can be projected whileavoiding the overlapping region in the projection range, and theposition measurement of the target objects can be accurately performed.Further, even in other measurement projection apparatuses, overlappingof the measurement light can be avoided, and the position measurement ofthe target objects can be accurately performed.

In the projection system, the processing unit 210 is configured toreceive an operation input by a user, select a measurement projectionapparatus configured to perform the mask processing and an error regionof a projection range of the measurement projection apparatus based onthe operation input, and set a mask region for an error region of theselected measurement projection apparatus. Accordingly, a desired maskregion can be manually set based on the operation input of the user.

In the projection system, the processing unit 210 is configured to inputa condition including predetermined priority information, select ameasurement projection apparatus configured to perform the maskprocessing and an error region of a projection range of the measurementprojection apparatus based on the condition, and set a mask region foran error region of the selected measurement projection apparatus.Accordingly, an appropriate mask region can be automatically set basedon the predetermined condition.

In the projection system, the processing unit 210 is configured to inputa condition including predetermined priority information, select ameasurement projection apparatus configured to perform the maskprocessing and an error region of a projection range of the measurementprojection apparatus based on the condition, temporarily determine amask region for an error region of the selected measurement projectionapparatus, receive an operation input by a user, determine the maskregion based on the operation input, and set the determined mask region.Accordingly, an appropriate mask region such as recommended setting canbe automatically presented to the user based on the predeterminedcondition, and a desired mask region can be manually set based on theoperation input of the user. Further, flexible and more appropriate maskregion setting such as a partial change, a fine adjustment, and areadjustment of mask region setting is possible.

In the projection system, the processing unit 210 performs the positionmeasurement at predetermined time intervals, and when the error regionis displaced, the processing unit 210 dynamically sets the mask regionaccording to a position change of the error region. Accordingly, forexample, when the target objects or the obstacle moves, the mask regioncan be set in conjunction with the displacement of the error region, andaccurate position measurement of the target objects can be performed.

In the projection system, the processing unit 210 allocates the maskregion to the plurality of measurement projection apparatuses in a timedivision manner by switching a measurement projection apparatus thatperforms the mask processing. Accordingly, in the projection systemincluding the plurality of measurement projection apparatuses, the maskprocessing can be performed alternately by the measurement projectionapparatuses, and the position measurement of the target objects can beaccurately performed. In this case, a measurement result can be smoothlyconnected at a boundary portion with a projection range of anothermeasurement projection apparatus, and accuracy of the positionmeasurement can be improved.

A projection adjustment program of the present embodiment is aprojection adjustment program configured to perform a processing relatedto adjustment of a projection operation of a measurement projectionapparatus 100 by a computer in a projection system including themeasurement projection apparatus 100 as a projection apparatusconfigured to perform position measurement and projection on targetobjects 105 and 106. The projection adjustment program is configured tocause the measurement projection apparatus 100 of the projection systemto project measurement light of invisible light onto the target objects,receive reflected light of measurement light reflected from the targetobjects, and calculate position information of the target objects basedon reflected light of the measurement light, so that positionmeasurement is performed. Further, the projection adjustment program isconfigured to detect an error region where a defect occurs in ameasurement result of the position measurement, and set, for a targetmeasurement projection apparatus 100, a mask region for a maskprocessing of limiting a part of a projection range when projecting themeasurement light by using a measurement result of the error region.Accordingly, when measuring the target objects, the error region of theprojection range can be appropriately avoided.

A projection method of the present embodiment includes: a step ofcausing a measurement projection apparatus 100 to perform a maskprocessing of limiting a part of a projection range in which measurementlight of invisible light is projected; a step of projecting themeasurement light onto a projection range partially limited by the maskprocessing; a step of receiving reflected light of the measurementlight; a step of calculating position information of target objects 105and 106 positioned within the projection range based on reflected lightof the measurement light; a step of determining a projection position ofa content based on the calculated position information of the targetobjects; and a step of projecting the content onto the determinedprojection position. Accordingly, when measuring the target objects inthe projection system, the error region of the projection range can beappropriately avoided.

Although the embodiments are described above with reference to thedrawings, it is needless to say that the present invention is notlimited thereto. It will be apparent to those skilled in the art thatvarious changes and modifications may be conceived within the scope ofthe claims. It is also understood that the various changes andmodifications belong to the technical scope of the present invention.Components in the embodiments described above may be combined freelywithin a range not departing from the spirit of the present invention.

The present application is based on Japanese Patent Application(Japanese Patent Application No. 2018-117208) filed on Jun. 20, 2018,and contents thereof are incorporated by reference in the presentapplication.

INDUSTRIAL APPLICABILITY

The present disclosure is useful as a projection system, a projectionadjustment program, and a projection method that can appropriately avoidan error region of a projection range when measuring a target object ina projection system.

REFERENCE SIGNS LIST

100 measurement projection apparatus

101 image capturing device

103 calculation device

105 first target object

106 second target object

111 lens optical system

112 infrared LED light source

113 display device

114 visible light LED light source

115 dichroic mirror

122 projection device

200 projection adjustment apparatus

210 processing unit

220 storage unit

221 projection adjustment program

230 communication interface (I/F)

240 display unit

250 monitor

260 input unit

401 image input unit

402 pattern decoding unit

405 coordinate conversion unit

407 coordinate interpolation unit

408 content generation unit

410 image output unit

411 pattern generation unit

1. A projection system comprising: a projection apparatus configured toperform position measurement and projection on a target object, whereinthe projection apparatus comprises: an invisible light projectorconfigured to project measurement light of invisible light onto thetarget object; a light receiver configured to receive reflected light ofthe measurement light reflected from the target object; and a calculatorconfigured to calculate position information of the target object basedon the reflected light of the measurement light, and wherein theprojection apparatus is configured to perform a mask processing oflimiting a part of a projection range in which the measurement light isprojected.
 2. The projection system according to claim 1, furthercomprising: a processor configured to perform a setting processing forthe mask processing by the projection apparatus, wherein the processoris configured to cause the projection apparatus to project themeasurement light to perform position measurement, and detect an errorregion where a defect occurs in a measurement result of the positionmeasurement, and wherein the processor is configured to set, for atarget projection apparatus, a mask region at a time of projecting themeasurement light by using a measurement result of the error region. 3.The projection system according to claim 2, wherein the processor isconfigured to detect a region where a measurement error occurs due to anobstacle in the projection range as the error region, and set the maskregion for the region.
 4. The projection system according to claim 2,wherein the processor is configured to detect a region where measurementlight from another projection apparatus overlaps in the projection rangeas the error region, and set the mask region for the region.
 5. Theprojection system according to claim 2, further comprising: a pluralityof projection apparatuses, wherein the processor is configured to: causea first projection apparatus of the projection system to project themeasurement light onto the target object; cause a second projectionapparatus of the projection system to receive reflected light of themeasurement light reflected from the target object; determine aconnection relationship of a projection range of the first projectionapparatus based on the received reflected light of the measurementlight; perform a determination process of the connection relationshipfor all projection apparatuses to be processed; detect, as the errorregion, a region where measurement light from another projectionapparatus overlaps in a projection range of each of the projectionapparatuses of the projection system; and set the mask region for theregion.
 6. The projection system according to claim 2, wherein theprocessor is configured to receive an operation input by a user, selecta projection apparatus to perform the mask processing and an errorregion of a projection range of the projection apparatus based on theoperation input, and set a mask region for the error region of theselected projection apparatus.
 7. The projection system according toclaim 2, wherein the processor is configured to input a conditionincluding predetermined priority information, select a projectionapparatus to perform the mask processing and an error region of aprojection range of the projection apparatus based on the condition, andset a mask region for the error region of the selected projectionapparatus.
 8. The projection system according to claim 2, wherein theprocessor is configured to: input a condition including predeterminedpriority information, select a projection apparatus to perform the maskprocessing and an error region of a projection range of the projectionapparatus based on the condition, and temporarily determine a maskregion for the error region of the selected projection apparatus;receive an operation input by a user, and determine the mask regionbased on the operation input; and set the determined mask region.
 9. Theprojection system according to claim 2, wherein the processor isconfigured to perform the position measurement at predetermined timeintervals, and wherein if the error region is displaced, the processordynamically sets the mask region in accordance with a position change ofthe error region.
 10. The projection system according to claim 5,wherein the processor is configured to allocate the mask region to theplurality of projection apparatuses in a time division manner byswitching a projection apparatus to perform the mask processing.
 11. Anon-transitory computer-readable storage medium that stores a projectionadjustment program, the projection adjustment program, when executed bya processor, configured to cause a computer to perform processingrelated to adjustment of a projection operation of a projectionapparatus in a projection system, the projection system comprising theprojection apparatus, the projection apparatus configured to performposition measurement and projection on a target object, the processingcomprising: performing position measurement by causing the projectionapparatus of the projection system to project measurement light ofinvisible light onto the target object, receiving reflected light of themeasurement light reflected from the target object, and calculatingposition information of the target object based on the reflected lightof the measurement light, detecting an error region where a defectoccurs in a measurement result of the position measurement, and setting,for a target projection apparatus, a mask region for a mask processingof limiting a part of a projection range at a time of projecting themeasurement light by using a measurement result of the error region. 12.A projection method comprising: causing a projection apparatus toperform a mask processing of limiting a part of a projection range inwhich measurement light of invisible light is projected; projecting themeasurement light onto the projection range partially limited by themask processing; receiving reflected light of the measurement light;calculating position information of a target object positioned withinthe projection range based on the reflected light of the measurementlight; determining a projection position of a content based on thecalculated position information of the target object; and projecting thecontent onto the determined projection position.